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HIGH-SPEED VISION SENSOR 



(57) A high-speed vision sensor includes: an ana- 
log-to-digital converter array 13, in which one analog- 
to-digital converter 210 is provided in correspondence 
with all the photodetector elements 1 20 that are located 
on each row in a photodetector array 11; a parallel 
processing system 14 that includes processor elements 
400 and shift registers 410, both of which form a one- 



to-one correspondence with the photodetector ele- 
ments 120; and data buses 17, 18 and data buffers 19 
and 20 for data transfer to processing elements 400. 
The processing elements 400 perform high-speed im- 
age processing between adjacent pixels by parallel 
processings. By using the data buses 17, 18, it is pos- 
sible to attain, at a high rate of speed, such calculation 
processing that requires data supplied from outside. 
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TECHNICAL REl n 

COOO,, T te p,.^ im , enfo „ reteles „ . Wg , speM vfe|o „ s . nsor ^ ^ ^ 

BACKGROUND ART 

the actuator can be controlled in units of milliseconds Acc^oTv nTi™ " ' ,S V ' Si ° n S6nS ° r and i,s actuat0 '. 
speed ,s required. However, because the image fJSL^SZ * n l^T SpeSd that matcnes «* 

be applied to such app.ications as image analysis HoTever the diJ £fh andpr0C6SSed Therefore., the devices can 
no, suitable for controlling robots or similar rea.-tfme a^pfcations ^ n0 rea, -" me Capabilit * and ™ 

Massachusetts .nstitute o, Technology, the Cal- 
the image reading portion and the proceJ^^^TlTrl^ a »*" «=hip that combines 

bedded imager," David L. Standley. SoUSl£\5£L Vol M 1 , n 'f P ° S,t '° n a " d Ori «**ion IC with Em- 
Motion Us,ng Analog and Binary tLmJiE^^^.*' Dae - »■ 1853-1859, IEEE); "Computing 
52-64. IEEE): and "Artificial Retinas - , as , SSS'fc^l HUtCh ' nSOn ' et al - Computer, Vol. 21 , March 1988, pp 
vember 10. 1994). However, these cl^^,^^^'^ ^ e ' ^uro, Vol. 372, No' 
c.rcu,ts have various shortcomings. That is the circu s^aui^ h " **** l ° integra,e Acc °^ingly, these 

-mage processing they can perfo'rm is limited to spec ia, "TT ^ ° f ° UtPUt Si9na ' S - The ^ of 

[0005] Japanese Unexamined Patent ApDlicat.on^ nhZt L ^ C ' rCU ' tS haV * a lack of universality, 
capable of perfonming universal image ££2£ tI^T* 0-145680 has proposed a vision chip that is 
Photodetector. An ana,og-to-digi,a, cl^ZZTci Z eaZZoZT ^ ■ P"*"-** each 
processmg time through parallel processing The vision* o can In ' , ^ ^ The V * Si ° n ° hip can reduce »e 
the photodetectors and the processing elemenSThS an ^^^"f 6 thenumberof transmission lines between 
[0006] It is noted that the center of n™Zm' t T 9 P mal lnte 9 ra tion 'evel for both, 
-age processings. In ^7£^2^J^« ™ ^ * that is required in many 

-age data of each pixe. and positional inflation % *^£ES£ " T" * t0 P6rf0rm C8lCUlati ° n ° n, ° 
mation) of the subject pixel. It is therefore necessZ to Sre T*T ,nformat,on a "d y direction positional infor- 
memory When using the technology in the JaSne^nexamTn^ information <* each pixel in a 

it is necessary to transfer, in advance pi^Trto™!^ f ^ Pub "<*tion No. HE.-10-145680, 
spending processing elements in MZ£S^^^J!TT B ? le, ' " M ^ t0 the — 
received from a corresponding phoM^^S^*^ Pe fT S . Ca,CU,ati ° n ° nt ° ima0e data that is 
of time to transfer the positional information t the procSna elem^c TT the , Ca,cu,ated res ^ « requires a lot 
a basic image processing operation. Processing elements. It is des.rable to enhance the speed of such 

DISCLOSURE OF THE INVENTION 

speed. dno wn,cn ,s capable of performing basic image processing at a high 

• -pln^a^ JMJJ;;;. invention provides a high-speed vision sensor, 

Plurality of rows and in a p.urality of co.umns an an l t dtaSf ^ * ^ arran96d ^^ensiona.ly in a 
converters which arc arranged ^-d^on^on ^ h ? * °» analog-to-digital 

in the photodetector array, each ^Z^Sco^r^n 7 c °™rter corresponds to one row 
successively outputted from the photode,^ \ ^Z ZZZT 9 ' ^ Si9 " a,S Which are 

« paralle. processing element array the paralSoLlli^T P ! 9 ^ 3 Para " el P rocessin 9 system including a 
are arranged two-d,mensiona,.y in a pSty ToZ ZTaZr^ T? * ° f Pr ° CeSSin9 e ' ememS Wh ^ h 
with .he p.ura,ity of photodetectors in the pSoSSSS^ e^ch L ° mnS , a " d ° n6 - ,0 -° ne correspondence 
c aI cu,ationondigita,s,gna.s which -transferred 
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ei-£g.JUL.a . 

spending row to the processing elements on the corresponaing row, ,-ianaisbased on the received positional 

portorrnaprcdetcn.lnedca.cu.ationtoca^ 

- the control circuit. *, ir +h«r rAmnriQP data buffers which are provided in 

providing the high-speed vsion sensor with the data buffers, ,t ,s poss.be ^ , ting t P ogether 

L data traLer speed, at which data is transferred ^^^^^S^^. *• register array 
I0 016] It is preferab.e that the parallel processing ^J^££SwdES with £ p lura ,i, y of ana.og-to- 
having a plura.ity of shift registers which *^ S ^ 

digital converters and in one-to-one correspondence with J™**^' analog-to-digital converter and 
successively transferring digital signals, which are outputted from the ~"~P«nd ng ^9 9 
which are equivalent to signals outputted from the P h "° d ^ to transferring 

high-speed image processing, and particularly real-time processing. 
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Hg 3 S a structural block diagram it Z" ZuJ «J » 8<,nS °' ' h8 ««<« 

amtodinrsnt; 9 °"'" g 8 e ° m, °' c " cl " 1 »">pte ys a ,n ,„e highspeed » lsion sens0 , M lhe 

S^^tEJESaKr*' ,M " — — «, pro,**, ,„ 

% ! 1 S^X^S*' *- "IN. a„a,cg.,c-d,gi,a, converts, array „ Fig 4 . 
sensor of the emOodLn, P"«~"0 a'ef.nt and . ah* register eraployed in ,te high-speed vision 

Fig. 10(A) is a gowchar, »"« •»» P* »'a s,ep S„o in Fig. 8: 

process ol Fig. 9. " W 0181 8um caK »'a»°n procasa In the canter ol gravity calculating 

Rg. tore, * , 0 , . tow , om cslcu|Mng ^ fc ^ ^ ^ ^ _ ^ ^ 

of Fig. 9- °' n81 * da'aalaang process ,n , he eent „ of gra>ily eabulallng 

the step S1 1 0 in Fig. 8- enn ^™nt process, wh.ch ,s one example of a preprocess executed during 

35 BEST M ODE FOR CARRYING OUT THE INVENTION 

™*£XiZZ2 SSnSOr aCC ° rdin9 10 3 Pre,6rred emb0di — °< * a P-ent invention wil, be described 
* Sg ^criS 1 ^ 1 ° f C ° mPOnentS deSi ^ ated b * the same reference numerate to avoid du- 

Ssa^exam^ 
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r0fl9ll c: '"c acti&or iu. 

Lrence to S F'i a S^SSr * - nigh-speed vision sensor ,0, w«n 

ana.og-to-digita. converter array 13 a oaraHei arJZtl n embodiment incl "des a photodetector array 11 an 

a y-direction data bus 18, a"Z^^ b ^T! T™. K 3 C ° nlr01 CirCUit 15 ' an **»"0" data bus 17 
and an output bus 155 (Fig. 6) toTZcZtZer ' " *" *** 2 °' an ins ^«ion/command bus 16,' 



"T00291 m k ycaunueu laier. - ,w > 

'-rCS 

are lined up in the horizontal direction (X direW^i ohZ^t t mC ' Ude N1 P hotodetec *°^ 120 which 

rows 110, which are arranged vertically (Y S , ^hjt ^ 11 inC ' UdeS h ° rUontal Photodetector 
[0023] The analog-to-digita.converter arrav^ Kr£?£ 0r,h , OQOnal t0 the horizontal Erection (X direction), 
dimensionally (vertically ( 'direction)) anatof Z h£ , analo 9 to di 9 ital converters 210 which are arranged one- 
with the N2 horizonta. *L££££ ^JhSjSSZ^ * ° Tf ° n6 - t0 -° ne — P-dence 

for successive.y converting electric charges ouToutted from h ' *** arate *to^Sital converter 21 0 is 

zonta. photodetector row 110, into yJS^S^J^T^"* ^ be '° ng l ° 3 c °" e *Ponding hori- 
digital signals. 9 S ' 9nalS and then ,urtner analog-to-digital converting the voltage signals into 

10024, The para„el processing system 14 is provide, with a processing element array 40. The processing e.ement 
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»» /dc> 4nn which are arranqed two-dimensionally (that is, N2 rows x N1 
array 40 indudes N1 x N2 processing eleme ^JStJSl^STL P«e«-no element array 40 is also provided 
columns) in one-to-one correspondence w h J^JJjJ^^™- data) in one-to-one correspondence with 
with N2 shift register lines 420 (l.nes of sh.ft "g^^^'S "SdudL N1 shift registers 410 (shift registers for 
the N2 rows of processing elements. ^^^^^^^^^^^^^fAv^^ 
ima ge data transfers). The total num^ shjft fegjsters 410 are 

[Sr™. -uit 15 is for controlling the J™^^ 
command signals and data thereto. The x d,rec^ 

respective columns. The x-direction data bus 1 7 is for transter ing a f respective rows. The y-direction data 

TheVdirection data bus 1 8 is connected to the ^TeXoTeZZ The S data buffer ?9 is connected 
bus 18 is for transferring data to each f^^^^^S^ data buffer 20 is connected between 
between the x-direction data bus 1 7 and the con^ol circuit J b_ v 1? . connected to the control 

the y-direction data bus 1 8 and the control circuit 15 In this way, the x diction oa ^ ^ ^ 

circuit 15 via the x-direction data buffer 19. The ^y-direction da »b«lB, co nrjjjj cjrcujt „ tQ ^ 

direction data buffer 20. The instruction/command bus 16 is for «^™8£ processing system 14. As shown in 
KTES? -™ ; .SS^ - the processing elements 400 via the s.ng.e 

Srw^hisconfiguration, N 2data, = 

array 1 1 with the parallel processing system 1 ^^"^ £ 2 ,„ a conf jgur ation that enables the operations 
system 1 4 can be formed on separate circuit ^£^^*^ Bpee6 JLion sensor 1 0 can be more stable, 
of each device to be verified separately. Therefore, P/°duct °n o the mg . sp jn 
Further, by forming the photodetector array 1 1 ^J^^S^^^^' " ls P OSSib,S * empl0y " 
way, it is possible to manufacture both dev.ces theS enabling a more stable production. The data 

processing method suitable ^o^rfensX^ ~* JjJ* « 9 fce formed on the same circuit board 

buses 1 7 and 1 8 and the data buffers 1 9 and 20 for the t and I y ai e » d on circuit boards 

with the parallel processing system 14 assho ^^^ 14 is formed. It is es- 
which are prepared separately from the c,rcu ^ 

pecially preferable that the data buses "anJMBand £** b ™ ^ between tne da ta buffers 19 and 20 and 
processing system 14. It becomes easy tc vision sensor 10 can be entire.y created 

™C^^ 

[0027] Next, the components of each circuit will be described 1 g g 

0028 Fig. 3 is a block diagram showing the ^TZ^OiseX -put/output interface 152. All of 

l CPU 1 50, a memory 1 51 , an irn 1 52 is for performing input/output operation 
these components are connected by a 1 53 .The ^^^ograms to be executed by the CPU 1 50. The programs 
with an external device. The memory 1 51 stores there n data ofprog am ^ ^ ^ pro 

include:, vision sensor process program shown ™° jma processing (Figs . 9 - 12, Fig. 13, Fig. 14, 

for controlling the parallel process.ng system 14 toexeoute a pa a. e g P ^ program 

shown in Fig. 1 , and the output bus 155 shown in memory 1 51 , the CPU 150 controls the photode- 

[0029] Based on the vision sensor process J^^^^^'ootbo. portion 300. and controls the 
lector array 11 and the ana.og-to-dig.tal "^f^^^^X* image reading control portion 300 (Fig. 4) 
paraHel processing system 14. More specially 

(during 81 01 of Fig. 8) to pick up images by us,r g the ph ^J^™^ rcgisters 41 0 in the parallel processing 
13. The CPU 150 also controls both of: data «* nsfeTS ° X J°^.°Z™* S102 S104 and in S110 of Fig. 8), thereby 
system 14, and calculations executed by the Processing < emen ^^sing The CPU 1 50 further performs required 
achievingasingleinstructionandmulti^ 

calculations while the parallel processing system 1 4 s exe cut mg tn obta|ned b tne paral | e l process- 
other necessary calculations, and ^^T^ ^ 1 50 communicates with the externa, device 1000, 
^^^^ —vice 1 000. such as an externa, actuator. For example, 
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reading centre, portion 300, in the control circu^ 5 ZSSZ^TT™ ° n, ° VOfta9e Siana,s - Th * '"age 
d.grtal converter array 13. The image reading ^ arra * 11 a " d 'he analog-to 

— S( gna (s , indict of operation Jngs. to ^^^^^XZ^^ 

a switch element 1 40. The Pholo^lS^l^K or" 3 Ph ° t0e,ectric e-ement 1 30 and 

mtensrty of received light. The switch element 1 40 is connected to » ? 9e i neratm9 an e,ectric charge according to the 
sion element 130. The switch e.ement 140 outpl t^e^ 

struck 30 ' S reSP ° nSe t0 3 h0riZontal *™ Z^vT^ nTb^T"^ m P hotoel ^ conversion 
structed from Nl photodetectors 120 which are arrant L ,k „ * hor,20ntal Photodetector row 110 is con- 

sents 140 being connected with one ZterT^ D Z^ t f ^' re ^) vvith their switch Z 

tector rows 110 which are arranged verS £ £ y Ztl *Z " ^ ^ N2 h ° r * 0n,a ' 

Thus, the photodetectors 120 if (i=i „ N1 hl'^.? ?' ' per P endicul ^y to the horizontal direction 

[0034] Next, the structure o'/the ana o Pto-dSL conSrl ^---"V in N1 x N2 rows ' 

■Sure 4. The ana,og-to-digita, converter C ^ * th * Sa ™ 

[0035J The analog-to-digital converter arrau n ,c f 9 process.ng unit. 

Each analog-to-digita. oo^2^£Z£ZZ£^ T converters 21 a G =1 ~ N2) 

todetectors in the corresponding horizontal pho^^ ^ 

parator circuit 230, and a ^ ^ "^"^ circuit 220J having a charge amp 221, a com- 
[0037] The integrating circuit 220 include- th« ^ r 

223, The charge amp | 21) is t^S^^^f" ^ Variable -P^tty unit 222, and a switch e.ement 
row ,10, The vahab.e capacity unit2 2 ? s electee on one end TT ,r0m the h ° ri20ntal PhotodetS 

•s connected, on the other end, to the output terS of^h V° ^ ' npUt te,TT1inal of th * charge amp 221 . and 

to the input tormina, of the charge J* ^ **> * » ™ end 

The switch element 22 3j is turned ON or OFF in response^ ItZ c ^ 6nd ' t0 the outDut te ™a' of the same 
not^ 

roSn C T C H Uit 220 Wi " bG deSCribed bel ™ torS c^uSi " ^ ' 4 " blt (18 9r3dati0n) reSOlu «°- Th * 'in*. 

of he charge amp 221 that receives signa.s output To^the cL^TT* °" 0n * ^ t0 ,he in P ut t6rmi " a ' 
of the switch elements SW11 - SW 14 fs connected betwTel ^ P honzontal photodetector row 110. Each 

C4 and the output terminal of the charge amJSf T^wttcn L l ^J* * COrres P° ndi n9 capacity element C1 - 
response to capacity instructions signal C t1 C 14 TespJc Tel EaS o/L 1 '■ SW14 tor ° penin9 and cl °^9 in 

LwT'sw^ SW21 ' SW24 has one 

SW 1 - SW14, and another terminal, which is connected to . cLunrt! , £1 a " dthecorres P ond ing switch element 
are for opening and closing in response to caoa^Ti N ° The SWitcn e,emente SW21 - SW24 

of the capacity elements <?1 - ^IJ^Z^ZZ^ ^ ' ^ Th6 



C 1 = 2C 2 = 4C 3 = 8C 4 

C o = C 1 + C 2 + C 3 + c 4 
[0040] Wherein C 0 is the maximum electrical 



capacrty required by the integrating circuit 220. It is assumed that the 
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. ^nt-i^n^PPPia 4^ is Qn and that a reference voltage is V REF . In this case : 
saturation charge of the light receiving element 130 (see Fig. 4) is u 0 ana 
the following relationship is satisfied: 

C 0 = Qo^ref 

tOO*, The remainder circuits in the -.og-^ f^J 2 

described below while once again refernng to F.g. 4. T ^^^^^ Mgi y^% x6 f W oxm^Q 
integral signal V„ which is outputted from the e 9 ra ^ 

. a comparison result signal V c . Based on the value of the companson result -JJ"«Vc. p » cjrcu|t 220 The 

13 can be configured to ha « « " pother ™ resoUrt.m ^ ^ ^ ^ tjrTijngs 

[0043] The image reading unit 11 , 13, having tne aoove w» readj comro| n 

by the image reading control portion 300 in the contro a eoniol signal portion 340. The basic 

300 incudes; a basic timing portion 310, ^^l^J^^^l^JL 11 and 13. The horizontal 
timing portion 31 0 is for generating a basic timing to atta, ^"J^i scan instruction inputted from 

tiVar,r:^ • - instruction si9nal R 

S5Titt structure of the par,, I P« system £^-£~ 4 js provided with N2 shift regi ster 
[00451 As described already, a» shorrrn in Fig. 1 , mo parai'sr P'<**>» •» ' _ _ N2| , „ pho ,odo4oclor 

La«0,0-1~N2>lnaada»ondcndaw^ 

array 11 'and in corraspond.nc. *«h th. « a „°' 0 l«w.d in aides. Each ehift register 

convener array 13. in each .Ml regis... iie 420,. N, Mi 41 r.g «^"<V» » nn ~ „ 10 „ ch 

SS?r«!S- Posing sys.en, ,4. M p-ceasin, . -££M - £ ! ™ 1 -532 
processing element 400 g (i = I ~ N1. l - ^) is co controller 15 is connected to 

\2%£ZZK£ttZ3!S^£ £ and coLis c_ «~ dy eacd 
processing element 400. 

[0047] Next, the output bus J55wHI be • J^ed tQ ^ sing|e output bus 155 Jhe output 

[0048] As shown in Fig. 6, all the proce «. n * 0 the out t bus , 55 is connected to all the processing 
bus 155 is constructed from a s.ngle one-bit W^'^'lJ^^^inmina to the control circuit 1 5, a logical OR 
e,ements 400, the output bus 155 f unct.ons as a , w.red f^^^S^ 15 can receive output signals 

,-.[0049] Next, the x-direction data bus 1 7 and the ^Mmtt ££££ be constructed from, for example, a 

on the corresponding column i. ctrlloted fra m a Dlurality of (N2, in this example) one-bit data lines 

[0052] Similarly, the y-direc.ion data bus 1 18 *^^J^t^ZJ^ elements 400(i.j) (1.I.N1) that 
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sponding row i. 

w £ d 7" a « 

><> register matrix 401 is of a 1-bit shift type. The register 2, 1 * t 4 X ^ and can be ™«>^ accessed The 
data input/output operation. Wore splcifica.ly, ^X^Z Z'TlT^ ^ ° f data St ° raae ° pera «<" and 
from the corresponding shift register41 0. a digital siqnll DlTa Z tn f Passing element 400 is for receiving 

nected to the register matrices 401 in fouradjacent prccessTno 1m ? V£ pr0Cessin 9 e,e ™nt 400 is directly con 
processing e.ement 400. The register matrix 4oT can^Zf °° 3re disposed ^cent to the subject 

matrices 401 of the four neighboring process* T^T* ?*" are St0red in th * "S 

nected further to a corresponding iS^cS^SJSTi^ SST ^ ' here ' n - ^ reQiSter matri * 401 j ™' 
d,rect ( on data line 1 80j in the y-direction data bus 1 8 rZ Z,^ ^ rection data bus 1 7 and to a corresponding y- 

s ^^^^ s,9n "* " aro 

element 400 executes various calculations by C^^ZST?"" " ** re9iStGr ^ 401 ' The P"*-** 
[0057] More specifically, the ALU 404 is a oL hit .? . , perfoim,n 9 th,s operation cycle. 9 
a logical sum (OR), an exclusive or (XOR), an SSJ^KS!!^ f ° rCalCUlatin 9 a P^ct (AND), 

can process only one bit at a time, the ALU 404 can execute ™ and like. Because the ALU 404 

calculations. I, is noted that complex calculation, ZTZZ^™. calcula «°ns by performing a series of one-bit 
calculations. Accordingly, the ALU 404 performs coZ£ caS^n* ? * C ° mbination of ,he above-described plural 
se ecting one function from these calculating fuS^S?^ ?TS Perf0miing calculati °»* each time 
t.pl.cation can be described as a combination of addLnc 5? ? Performed by the ALU 404. For example re- 
peating adding functions a plurality 0(5^^22^^^^ T can perf °™ multiplication by r e- 
o be subtracted, adding one (1) to make the value negate and then n i r6VerSin9 ^ n ^ ber 

mplemented by setting the divisorto a number, such 2 8 4'or P an S°T 9 add '' ny ° Perati0n Division ca " be 
three places to the right if the divisor is 8.) An absolute vafu Xr a ine^ 9 (F ° r 6Xample ' the bits are shifted 
sign of the negative number, whose sign bit is 1 that i H I 9 ! * nUmber Can be ca 'culated by reversing the 
0058] As shown in Fig. 7(A), in the'regfe er matr 7^Z^ " 5 3ddin9 ° ne < 1 > to tbe ^ 
function registers 4012. The 24 one-bit registers 4010 can b2 ™Z? ^ 24 r6yiSters 4010 and eight 

are for enab.ing input/output operations wfth external deWces aTZ^ZlT' ^ "*« 401 * 
these registers are treated as defining a single address soar! Wrtn " e, 9hbor,ng processing elements 400. All of 
m F,g. 6 indicate the addresses assigned ^^o^^^T^ * the reyiSters 4010 and «>12 

assigned addresses 0 - 23, while the'eight SSk^S^S^ «* c " te ** the 24 Asters «>"> are 
ration, input/output data can be accessed in the same man™ « Z ^ addresses 24 - 31 . With this configu- 
[0059] As shown in Fig. 7(A), the register TamZTtZ^T 9 ' r0m and Wri,ind data int ° tb * agisters 
of these 32 registers 401 0, 4012. AH ofthe renters "010 and 25 ™ ° R 9316 4 ° 14 ' WhiCh is connected «» a » 
403 v,a the single OR g a , e 4014. When one address in the reailf are "°" nected t0 the A-latch 402 and the B-latch 
register outputs its content, whi.e all the remaining ^Zse ^Z2?Jfa t T ^ * Se ' eCted ' ° nl ^ the se,e ^ 
onthevaluesfromal, the registers isoutpuned 

;=nrir s rr 2 s^ 

from , h e shift register 410. The function register 4012 aTaXL T 41 °' a " d iS USed for '"putting data 

401 of four neighboring processing elements 40^1?^^"^ T t0 r69iSter matrice « 

respective.y. of the subject processing element 400 The funct^ 1 ^ °' " P ' d0Wn ' ,eft and "ght, 
used for inputting data from a register matrix 401 of a Lr J" h re9 ' S,er 4012 3t SaCh of the addresses 24 - 27 is 
40a The function register 4012 at address 28 i ^SSES*Zt ^ST n6i9hb ° rin9 Pr0CeSSin9 te 
to the contro, circuits is noted that the — ^1^^^^^^^ 
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.000,) areconnected to thesing.e output ^^^^^^^^Z^t^ 

spending y-direction data line 180j ^f^^^^^^Z^ x direction data line 170i in 

data line 1 80j. The function register 401 2 at address 31 s connec ted t a P 9 ^ ^ ^ functjQn 

tne .direction data bus 1 7. and is used ,onnputt,ng dat « ^ read data . 

register 4012 at address 28 ■ .con [if^llr, 401 and ca.culations by the ALU 404, thereby 

[0061] The contro. circuit 1 5 "^"^^d InTuttoulput operations of the processing element 400. In order to 

controlling the entire processes of calculations and mpuuou p P ^ ^ examp|e fta conUo , 

write data, inputted from the shrft register 41 0 to add es z o (0) ,n g qr ^ q 

circuit 1 5 outputs, to the processing el^ent 400 ar , ,ns J^t» n » ister address (0 ). It is noted that 

and the sensor input (register address ^ n ^^^^^ he va iue four times as long as a basic Cock 

[00.41 F« M reading ^ ™ T„S - >»e MM SW21 ■ SW24 ,o OFF W«h 

SW11 -SW14 in ihe variable copacily una 222 ol Fig. 5 10 un a™ „ set to Co At ihe same 

f„™ operation, ma cap,* fcatwae n ma input ^^^^S^.^ V, * satte a state ma, 

r/r.t™r.rr2;:Pr^ 



20 



25 



30 



V S = Q/C 1 



35 



40 



[0067] Theintegra,s,gna,V s isinpu«ed^ 

1 Vref . If the difference between the integral I s.gnal V s ^^.^^Xo match the reference voltage V REF . In 
that is, lower than or equal to ±C 4 /2, then the '^J^^SElS.. On the other hand, if the difference 
this case, no further capacity control • performed and the «^°P^ be perf ormed t0 continue the integral 
does not fa., within the range of resol ut.on^ then ^^^^^ 240 opens the switch SW22 and 



45 V s = Q/(C 1 + C 2 ) 



50 



55 



^Mit p^o . Ra 4} and is compared with the reference 
[00681 This integral signa. V s is inputted into the "J^J^f^^ mechanism 240 opens both 
voltage V REF . On the other hand, if V s is smaller than V RE f ; .then me cap V ^ & . g ^ y& 

switches SW11 and SW22 and subsequently Coses the switches SW12 and SWZ 
changes to have the following voltage value: 



V s = Q/C, 



_ Tnie a, 9 „a, v s . oetpetteO » « — 230 »• compare* „i,n m. rn.nrenn. ooitage 

By W a,,n 8 m,a ,.er*acK .op among M M circoa 220. ,0. campara.o, M 230. »e cap.* 
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S£!*Sr£ a s ' 9nal M °" " tSS?. a L a ,,me Tte 

U/C1 This integral signal v s is comparee with the mr.JL.1 . . e6y makl "9 irae 9™i signal v« eoual lo 

a.grritrcant bit) » the integral signal v\ is IT, f W * °™ '" iS ou,pu " efl <* '»« UtC 

» (meet signilleant bit) it the imeor.l aT_.. ™ vMa 9 e V REF- A Zero (0) is OUtDUtterf « th^ i^od 
the integral signal V s be equal ,o ojci + C2) when MSB f TZ^I 02 is set * ON making 

= 0. Th.s mtegral signal V s is compared to the 7elZnZlZ, ' . * ' nte9ral Si9nal V s be equal to Q/C2 when MSB 

cempleteg, ,h. ^^mJ^^^^STTr "° m »» '"»' 

eepeerty una 222 » Subsequent* Xil™? " abted * 9 ""' "Wily value ot ,„. l!°lt,° 

successively from the shift register 41 0 at one ohcehc S S 9 " re9 ' Ster ' ine 420 ' ^ transferring the digital .£2 
410 are provided in the present m*^^^ , S^^ 410 ^^P^. BecauseL sh fregS 
mdependently from the catenation processing peCTd K^thT" ° Perati ° n by * he S ™ registers ca " ^ peSed 
^perform a pipeline process to control the SK^^wJST? 4 °°' Acc °rdingly, » is pos^ble 

elements 400 to perform calculation 9*om^S£££ 0 £?r™ "*? data wbi| e -"trolling the processing 
he processmg e.ements 400 at a higherframe rate t is notelauhlhi 6 P ° SSib ' e l ° Perf0rm calculations at 
to-chgi al converted data in response to a transfer start Inaf wh^h Sh '?/ ea,sters 41 0 begin transferring the analog- 
410 return a "data transfer complete" sional to thiT 9 , '• h ' S SSnt from tne con trol circuit 1 5 The shift reai* °,? e 
the number of bits that is equa. to 2 Zuo^ZXtZer^ f & ^ °PeZnZ 

data > levels. Thus, an efficient transmission ope £«o ca XZZSZT* 3 f ° W a " d the number °' «*S of analog" 

stored in the register mS^S fo nV^2i^. P, T B ^ 6,ementS 4 °° 38 d ' SCribed ■** Signals 
elements 400 when required. Data and ^Z^gnals aT S, 4 °° - tranSmitted t0 «"» Processing 

elements 400 via the x-direction data bus 17^nS3r^^2^^ °~ ^"'^ Circuit 15 ,0 tne Processing 
each processing element, signals for calculat on! T US 18 ' and ,he butfera 1 9 and 20 Thereafter^ 

B-latch 403. Predetermined 1,1?^^?^ 4 ° 1 "» tta A " ateh ^ 

[0076, ,t ° UtPUt 155 t0 ,he COn ™ otulV ^ Ca, ° U,ated rSSU,t fe via ^e register 

can be performed at an extremely high rate o soeZ l ^tZZ T Process.ng manner. Accordingly, calculation 
are obtained by the paral.e. processing system 14 tt'^lt T™* 15 ° UtpUtS ,he resu,ts oi calcu.a ions w S 
external equipments. For example the r2ute f ^,!, , "! ^ 1 °°° SUCh as an exte rnal computer or other 
used as OIM/OFF signals for Cjn^S^^ S ^ Para,le ' ^tem U can be 

on the results of catenations obtained by the para.te. P rLZ Z Zl TlT™ Ca ' CUla,i ° ns based 

9 System 1 4 ' and then output the calculated results to 
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the external circuit 1000. arfrtrrri0 rt hv the oresent vision sensor 10 will be described with 

Kir*: rs^-7=s^ ~sr-!=- - - — — ~~ up 

thereby successively switching the reset s.gnal R the snav pho todetector array 11 , 

cessively switching the h ° riz °"*^ photodetectors 120, (hereinafter 

he corresponding shift registers 41 0. This transfer proce* ^7J n ^™ r ^l TBdtoa8(Xiy , ) . 

120 reaches the shift register 41 0, at *e correspond.ng ^ M JSS^ transferred from the corresponding 

[0080] Aftertransmission has been completed in S1 04 , data i(x, y £ ? (hereinafter referred to as 

shift register 41 0 to the register matrix 401 c ^» J^ 9 4 f 0 Toompoii of a plurality of bits (four 

processing element 400(x, y». As shown in ^'J^^^^r 41 0 one bit at a time into the register matrix 

bits in this example). Accordingly, data is transferred from th ^MjjJ«4Ki processin g. 

401 . In S11 0, each processing element 400 is < ' c " 1 ^^^^ [i^corre^ponc^ng processing element 400, while 

[0081] When each shift ^410oon^ l«n^^^ P 410 continues 

Uch processing element 400 is -^^^^ a ^S^ 'are executed again. That is, the photode- 

to the process of the next frame in S1 05, and the P r0 ^ esses 0T ° ft jsters 41 0 are controlled to perform 

tector array 1 1 , the ana.og-to-digital f ^"Ll'o the pa^p£o~£g performed by the processing ele- 
inpumransfer operation for the next .rame^ h e^ 

mC nts 400, after the processing for one frame S 10 . transmitted in S1 04 from each shift register 4 0 

trame process in S106. As a result, 'W^^^^LS" Then, the parallel processing .or the next 

ously, thereby achieving calculation processings very rapidly. 
(Example 1) 

[0083, According to the present embodiment, SES 
processing element within a short period of T^^^^JT^^ P-ceLing element 400 performs 
outside. Accordingly, calculation can be attained at a ^^'^^ttematJForexample. in orderto calculate 
inS110atypeof calculation that requires add.t.onal data othe ^™** e ™°™n the processing elements require 
the center of grav.y of the input image ^^^^SSS^Z^^ P-essing elements, which in 

is calculated during the image P"^""™^^^ 15 previously stores therein x-direction positional infor- 
[0084] It is noted that the memory 151 .n he 1 ^ and y y djreclion posi tional information (which will 
mation (which will be referred to simply as x address J™ n «J^ co y ncrete terms , the me mory 151 stores, as x- 
be referred to simply as "y ^\ t ^^^T^S^ address, binary number data indicative of »V - 
address, binary number data indicative of 1 - N1 , and stores, as y 

" N2 " , th» renter of oravitv is based on a calculation of a weighted sum in each of x- 

to image intensity data l(x,y) at each pixel posit.on (x,y) of the subject image. 
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(x,y) by its positional information (^KSSSSS ^ e,ement 400 has ,0 mu '«P'V -age data 
together. Accordingly, when it is desired to cSatell ^ 0 " P ' Ura ' Posing elements 40oTe added 

'"^ance.toeach processing e.ement 

olthT^nT":^*^ •^ h ^^^-^ng ( S^i^S2SS^ na, inf0rn,ati ° n - ThSn ' a " X ' di « 

«?im .Lh I ha ' ' ne ,ma 9 e inlensi 'y sum "1 z l(x »>• is calcuiail h 1 (X,y) by a " ,ma9e intensit y sum 
Similarly, in order to determine a y-coordinate of thl tlZ calculate <l by add.ng image intensities of all the pixels 
is first ca.cu.ated a, each pixel by LS^^^^J^^ T a V«'«*™* momen y • y 

The y-coord,nate of the center of gravity is calculated h„H h 9 u t ° gether the y-d'rectional moments of all the pixels 
-mage intensity sum "ZZ\ (x, y) - * ca,c "'ated by dividing the y-directional moment sum " I z y . ,^, y) - bytne 



30 



35 



[0088] 

execute 
(Fig. 8) 
[0089] 

in S350 
the ceni 

«• — ,33,0, „ « g , h , , mage ^ sum £ r |(x y) ^ b< ^ ^ ^ b 

[0093] The sum calculation process f^am «,iii h -T « wnsraes i(x,y). 
« [0094] First, in S1 002 objective data S a*™** with reference to Fig. 1 0(B) 

is read from the register mS^t he A ,aThT 2 7hT *? ,(X ,' y) ' WhiCh 3 SUm to be determined) 

from the register matrix 401 to the A latch 402 Ne* In S1 004 dataf ^ U ° m itS ,east signS bi 

matrices 401 so that each processing element 2 !! ' S ™9hbohng registe 

processing element 400(x + 1 , y) ,hat is toca^ S^Z^TZ T * "* ' <X+1 ■ »> f — mother 

he x d.rect.on. The processing element 400(x y) store ,Z1 1 ( ' V) ad,aC6nt t0 the P*el location (x y) in 

he transfer function of the register matrix o, etc ^STJlSS? ^ ^ by 

transferred to a vacant region in the register matrix 0 I h 7 f ne, 9 hborin 9 re 9'ster matrix, an image data value is 
m SI 006, the objective data (image dS~1 vH IS hTt ^ °< lhe ^ighboring pixel Next 

read into the B latch 403. In S1008 data in A latch 40? J? transferred from the neighboring pix"x + y) is 
js stored in the A latch 402 via the re^, -£S?^ ^310^'^,^ ™ e «^ - 

herem a sum of the objective data of two adjacent pixll n thll T 12 th ' S Way ' each P ixel *°"(^) stores 
10095] Next, in S1014, the CPU 15 initial ly sete one m i « k," 1 " 6 ' " SUm ° f ' ma9e data '^'V) a "d l(x!l y)) 
x d,rec,ion. In S1018, transfer between nrt^i^ii'JSf*." 1 ^ Wi " be used l ° a sum in the 

processing element 400(x,y) receives a calculated ^1^^^ * S Perf0mied »*» 80 each 

processing element 400(x + 2. y) that is located at a ^«Ii ' ' (X+2 ' y) + l(X+3 ' that is stored in another 

closest to the subject oix e . nnc,^ , " LV ™ p °l mon ( x+n > V) (where n = 2': in this case, n = 2 1 - 9\ comn^ 
Ccuwed ,., u „. Next. m. ^J,«^ tt *'r; l »» • l «~"t 400,x,», s.or.s ,he,e« ,h. ^ 
. - - „. B , 8Kh 403 „ s , 020 . Data ,„ „a ?T ^.^^^^'1 
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S102 , The added result is stored in the A latch 402 

each pixel 400(x.y) stores a surr , of image data (^y) x y) , I 0£ £ and ljx+3, y) 9 ^ r = ^ ^ 

[00961 Next, i is incremented by one (1) in SI 028. it is juags d the va|ue N1/2 

L ha.f N1/2 of the tota. number of J"* « register matrices in x direction 

(no in S1030), the program returns to S1018. In S Ll a element 400(x y) receives a calculated result 

s performed n times (in this example, four times) so that ^W^J^ a position (x+n . y) (wher e 
(Ige data-added resu.) that is stored in anot her ^1 The transferred data 

n = 2'. in this case, n = 2 2 = 4) that is n-th closest xo me st j y y per f 0 rmed in S1022 - S1026. 

is read into the B latch 403 in S1020. Adding , operation the sam a. J^J^ol sToiB - S1030 are repeated. 
[0097, Next, while sSnth (n = 2*) neighboring pixeUhe 

Thus, addition is repeated through the eighth (n z r n g 9 h neighboring pixel. As a 

32-th (n = 2*) neigh boring , pixe t h. (n - 2 ) ne ghbonng p* ^ p ^ ^ ^ _ eiement 

S?.J? J^SSir.TSS -ge 'of l: subVct ro^. When n exceeds the va,ue N1/2 (yes in 31 030), the 
program proceeds to S1 032. 0 „ rt00 th« i ma n 0 intpnc;itvsums which are obtained at the respective 

n v di.ection is performed so that each processing element 400(x,y) ece.ves an aoa 

processing element 400(x, y + m) that is located at a P^Jj^ JJ" * * " * ^1^0 added result' Next, 
E> the subject pixe. position in the y direction. The J 8 ^ h ^ ^ jmo the B latch 403 in 

,he calculated result data, which has been ^^^^^^SS^l S1038. The added rosu.t is stored 

~nsV^= 

in S1048. Thereafter, the program returns to S1034. performed twice so that each 

above is performed in S1038 - S1040. processings of S1046 - S1040 are repeated. 

[0102] Next, while successively incrementing j by one (SI 044) tne p g ^ 

Thus, addition is repeated through the fourth r-£^S^W^5nJL!« pixe? ... up to the (N2/2)- 

ES^M- >« P~ eeteco.r, r~-OT>, - ^rcluT^rStnoan Brecon 0„a ,»e 
101051 During the pixel selects process, «*. « S1102 the CPU ^ „ us „ ,„„ 

Th^U 1 50 transfers data (0) to a,, the ^^ZVZZV^, Z f ilter matrix 401 , data that 
[0106] Next, in S1104, each processing element 400 x y) ,s con ^ e « amo a|| the processing elements 400, 
is transferred from the corresponding x .'"^^^ 1 ™ ^ S y V N2 ) whose x addresses are x1 (=1 ), are stored 
the register matrices 401 at the processing elements WM-J-*^ with data (0) . Nex t, the 

with data (1). The register matrices in the other remaining processing elements auu 

thus stored data is transferred to A latch 402 in S110* direction data | ine 180] (which will be referred to as "y- 
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objective pixel position (xl.yj) TheCPlimntw , 

!— , ,,om - -^Xss^sr^sg i?rr ,o s, °"- - * « <*« 

value in B latch' n 403 a ^ALU^Sp 9 IT™ 4 ° 0(Xy) * COntro,,ed t0 ™<«Ply the value in A !■*.„ ^ 
a = 

the multiplied resu.t obtained in £ ^ a IT, 1 '* Com e |eted ' the program proceeds to S372 (Fin irvAu i~ ^->~ 

B ZTZ 9 '^ mmriX 401 10 the B ,atc " 403. Ne^ fhe ALU 4^ rl t T Pr ° C6SS (S360 > is transferred 

to t sum °? " ? 37 f : AS 3 reSU,t ' at lha Pressing e^eT 4 oZ7eS'7 A ,atCh 402 and the 

ssT r resu,ts ' which are ° btained ^•»12s na '? 401 to the output bus 155 - As - -u r 

155. , n this case, only the processing element 400(1 ) a, thTi! J 9 tS ' *"* ° UtpUtted t0 the single output bus 

S«x.y) as the multiplied result, while the other remaining o^i 9 ^ P ° Siti ° n 0Ut P uts the ™«ge intensity sum - 

results. Accordingly, the output bus 1 55 U^Z^ZTT 9 ° UtPUt *• 2er ° val ^ (5 as the IS 

2 r rzt of sra " ~ ~ p ~ (s4 ° o) - * — - ~~ ■ 

VJ of each pixel. ^^SS/JS.SSS INSISTS ^ ' X ; direCti ° n P ° Siti ° nal <" f °™™ C* address data 
x-d.rection positional information to the co^dZ 'n n thS Center of 9 ravit V- ™e CPU 1 50 trans era eac h 

by temporarily storing each x *^£Z$^*Z^*^ ^ ThiS tr3nSfer 0 " era «° ^ Performed 
data v,a a corresponding data line 170x in the x dir^ion 2 T T-?^"" 19 ' a " d then herring the x address 
t^ms least significant bit one bit a, a tiJla^Ta^ZL X data iS transfe " ad 

or all the p.xels that are connected to one x direction date Z 1 Jn t^*' ^ ^ X address data is assigned 

L 8 T !° a " the pr0Ces -9 e-ements 4oo x y th" 1 ItSed E J""' "* *" 1 7 °* Can tra -" " 
[01 13] in each processing element 400ix v> ? rated at the corresponding x address 

S404. T eafter; the ad(jre d ss XS'SS STT, " X " 18 St ° red in " hS ^"«* 401 in 

(x.y) is aga,n read from the register matrix 401 into ,L a , ? A ™ rtS ,east si 9nif'cant bit in S406 Imaoe data i 
402 and data in B latch 403 taS4ia^ m S£S esuC S fJ" S4 ° 8 - Th6 ALU 404 mu,ti P'^ -bSTa la tch 
he reg, s ,er matrix in 8412. mU,,lP " ed r6SUlt ' that ls - the x direction moment value x . I(x,y), is stored il 

r 0 ; ;s ; ^ s ™ tota ' sum ca,cu,mion process is execu,ed to determine - — 1 — - - momems 

efer^ 

[0116] More specifically, in S1002 at each omLt n ^ Va ' Ue X * as ob i«*™ data 

y) for which a sum is desired to be de^e'd °is Z^n^ HT*" ^ X direCti0 " mome "' * • <(x 
S1004, each processing e.ement 4O0(x7) rlc^To^ZV^ ? re9iS,er matrix 401 to the A latch 402 Next fn 
processing element 400^1 »\ .h . , rece,ves objective data in this example fx+11 . 1^1 „«, ' 

the x-direc 9 tion m^?^ f.'f ^ JJ^rTT? P ° Si,i ° n <X+1 ' ^ £ X di ^ « " ^Tn sC 
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,(x,y) * (x + 1) • «x+1 V)) of x dire* *n , momen t *. * * £ *^ ^nsfer/adding process is performed so that the 
[0117] Next, i is initially set ,n S10 14 to one IV In SW« Slogans jJ h P anotnercateulated resull ((x+2) 

calculated result (x • l(x.y) + (x+1) • Kx+1 , y» at each P roces *3 e ™ n ^ , tnat is located at a position (x+n, 

„ > / ox . ii.^ that i<! stored in another processing element 4UU(x+^, y; inai i^a^ k 
. .(x+2.y) + (x+3) • I x+3 y» that . st ored in an p ^ jn ^ x djrectjon 

y) (where n = 2', in this case, n = 2 - 2) second cto» „ 10J)m \ he Dr ocessinqs of S1018 - S1030 are repeated. 
[0118] Next, while successively incrementing . by on SI 28), £ e P™^* theeignth(n = 2 3) neighboring 
^hus, transfer/addition operation is repea^ 

pixel, the sixteenth (n = 2*) neighboring pixel, the 32 th (n - 2 ne.gnoo n g p } + + N1 

up to the (N1/2)-th neighboring pixel. As a result, a ^ of a H the x dn e cho "^JJJ^ ^'corresponding row. 

. I(N1 ,y), in each ^ Y \^ S ^S^^^^^ ^ iS S ° 
[0119] Next, | is miually set in S I032 1 zer > <^ n 5 ™ gdded result jn anotner processing element 400(x, 

STon"S rSS^Con ( x, y+m) (where m = * in this case, m = * = 1) accent 

Thus, transfer/addition operation is repeated through the second (rn i J ™» Dixe i the 32-th (n = 2») neighboring 
boring pixel, the eighth (n = 2 ? neighboring P"£^Jg£ =f g ^Taretut a sum X i x . 9 |(x,y) of 

processing element array 40. Then, ^E^ESS^ ^ ^ -eds to S430 as shown in Fig. 11 . 
!oi 2 3, D uringthepixelse,ection = 

r"e a Si^ (0^ * ,, th other ^^^^^ as shown in R , * .. 
0124] When the pixel se.ection process (S430) ■ • JgJJ E^ed from the register matrix 401 

In si the multiplied resu.t obtained in the ^^S^SS^t^ tota sum calculation pmcess (S420) is 
to the A latch 402. Next, in S434, the calculated res « tt ^ a J «J t ~,J ™ mu|tjplies the value ,„ A latch 402 and 
transferred from the register matrix 401 to the B latch 403. Next, the > ALU 404 m i P (1 , 1 ), an x 

Ine value in B latch 403 in S436. As a resu.t, at the ^^^^J^SS^z x . I(x,y) and the 
direction moment sum I Z x • l(x,y) is obtamed as am* Jed '^ult °J the ca,c ^ & 

value (1 ). On the other hand at the othe, ^^^^^SjSS is stored in the register matrix 401 . 
result of the calculated total sum and the value (0). In S43* trie mump bus 155. 

,n S440, the multiplied result is outputted from the ^^^^'^ZSI.Sl. are outputted to the single 
As a result, the multiplied results, which are obta.ne Ut al the p oc ess W outputs the x direction 

output bus 155. in this case, only the proces s,n g elemen 4 ^ 1) JJ2ZpS«-i5 elements output zero values 
moment sum £ £ x • l(x,y) as the T^^^SJST^S^ circuit 15, the x direction moment 
(0) as the multiplied results. Accord.ngty the output *»^™2Sated at all the processing elements 400. 

ESnJISJ as described above for obtaining the x ^^^^^Z the memory 1 51 , y-direction 
[0127] More specifically, as shown ,n Rg. 1 .Z. f irst, in SB02/ neu . direction positional information 

'positional information (y address data "y") ° ^"^na S^em wtth its y-directional positional information. 
rotheproces S lngelements400soastosupplyeach P rocess,ng^^ y data fauffer 2Q 

This transfer operation is performed by temporan.y ^^^^^^^^^^^^^ 
and then transferring the y address data via a correspond >« *ta ta» 1 80 " ft. y n ^ ^ ^ 

data is transferred successive^ from its least signrf cant b,t one brt at a hme , _ ^ 

..m,^^^^^^^^^ N2. According* each data line 180y 
data is data of a binary number that ,s mdicafye of erthe one of values of corre sponding y address 

transfers the log 2 (N2) number of bits that "^^^^ ^1 the least significant bit. Thus, each 

xs rsr r; ^ - " P rr 9 « a re ^ - - 
[o^r-fc^ 
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£S££L SI? in 351 0 The calculafed •— • - - - v moment vaIue , . ((x y) is stored , 
5 £.<™ 
-.eL^ 

[0131J More specifically, in S1002 at each d^JLJ ™ ? m ° men ' Va,Ue V ' ' (X ' y) as ob > ect ™ data 
y) for which a sum is desired to t^S^X^ IS****" (3 y directi °" va.ue y . , (x 

SI 004, each processing element 400(x yTrSes oES T register matri * 401 to the A latch 402 Next in 
» .ng element 400( X+ 1 , y) that is located a ESSJKST h,S ? Xample V * ' (x+1 ■ *» from «noth» proces 
d.rect,on moment value y . , (x+ , , y) is ^ h' V> ** X direC,i ° n Next < in S10(£ the y - 

dHM.cn moment va.ues in the two adjacent pi e are aSe ftSS" *5 ? A latCh 402 and data in B 403 y 
402 via the register matrix 401 in S1008 - S1012 in^L 9 u ' a " d the added result is stored in (he A lateh 

.(x + t 2 y)) of y direction moment dataTtwo'Snt pfceT ^ a «" * ' V) ^ 

i^^v^XE "sr r^r D r 01 8 - si ° 26 ' transfer/add ^ * so « the 

(where n = 2- in this case, n = 2 i = 2 ) second closet to ? °° (X+2 ' X) that is located at a position (x + n v 
[0133, Next, while successivety In^Z^^Ts^T^ ^ "» X 

Thus ransfer/addition operation is repeate dthro ugh" I^S^LET^ °' S1 ° 18 " S1 ° 3 ° are re P eate * 
Pixel, the s ,xteenth (n = 24, neighboring pixel the32-th7n -Sf nl^h ne,ghbonn 9P lxe| . '^eighth (n = 2*) neighboring 
up to the (N1/2>-,h neighboring pixe,. As a result a urn «££?Z5Z* *~ ( " = **> 
(N1 ,y), in each row is obtained in the n mM «i , y d,rect,on moments, y • 1(1 y) + v • K2 V ) + ^ „ . 

addsd M obtained at each processing el.rn', " ! ' ° 40 ' »•*<«»•« prooos* is pedodned so that an 

bonng p,xel, the eighth (n = 2 3) neighboring pixel the s^mhTn ^7 ? neighboring pixel, the fourth (n = j*) neigh- 
p ^™*Asa^ 

in S532, the ™.t^^ P«"™ P-eeds to S532 as shown in Fig. 1 2 

401 to the A latch 402. Next, in S534 the Sated tluS T* 8 (S530) * ,ransfe "ed from the register matrix 
is transferred from the register matrix 401 TS^S^^^T^ ^ ^ Ca ' CUlati ° n pr ° C6SS ^ 
the value in B latch 403 in S536. As a result at the ™«I , U 404 multi P |ies »e value in A latch 402 and 
direction moment sum Z E y . (fx vl fe JStalf P rocess '"3 element 400 of the leading end address fl n a „ ! 
value (1 , on the other ^ SS;^^ -.cu.ated tota.lum X x i ^ 
resu, t of tne calcu(ated tota| " 9 P ^ »ng , elem nts, a ,ero value (0) is ob.ained as a multip.ied 

•n S540, the multiplied result is outputted from the ou pufaddriss SfS? " ^ matrix 401 

As a result, the multiplied results obtained at all the d«^Z , re9 ' Ster m3triX 401 t0 ,he out P ut bus 1 55 

bus 156. .n this case, only the processing e^tS^Zt^ "J*™ *" ° UtPMed t0 the ^ output 
sum r Z y • I(x,y) as the multip.ied result while Z other '.lit 9 e " d P ° Si,i ° n ° UtputS ,he V dir ^ion moment 

mu.tip.ied resu.ts. According.' the outpu b * S " ZZTr 9 GlementS ^ *"* Va ' Ues <°) - S 
• I x y), which is the .ogica, OR value of the resutts S^ ^H th C ' rCU,t 15 ' ,hS V direC,ion moment su "^ = ^ 
[0140] Next, in S550. the CPU 1 50 divides the7J l ^- * P rocessin 3 elements 400. Y 

sum Z Z , ( x,y,, thereby obtaining 1?^^^^^^ " V * ,(X ' y) by ^ ^ ^ 
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pixel <(x1 . ,1 ), for = »> i» *J« yiT white date (0) is „ansle,red llvopgh «,e e«, remaining lines. Each 

„„, ,e,.,enne,e Fig- 1 Of. n ™|™'£ - — - * V> 

r0146] More specifically, in S1102- SHOb, tneuru louua \ / CPU 150 transfers 

with itl x address being the same as the x = * 0*0 ve P« ^oC^ 
d ata (0) to a., the other « »n data hne £ Ox x , x )Jn^ £ ^ ^ ^ data (1 ) The Qther 
400, only the processing elements 400(x1 . y) (i - y _ w transfers data (1 to a 

remaining processing elements 400 are set w.th data (0). Next m S1108 S1 m 2 the CP 

y . dire ction data line 180y (y = yD with Js y ( address being J^^^lL 180y (y - yl). In this way, 
(x1 , y1). The CPU 150 transfers data (0) to all the other remaining V addresses are 

amona all the processing elements 400, only the processing elements 400(x, yl) ( I = , x _Ni). wnos y 
^ -t wYh data (I^The other remaining P~^^^^ x direction 

see ^i^^ - « * — in a " the 

other processing elements 400. comp , e ted the program proceeds to S612 (Fig. 13). In S612, the 

[0148] When the pixel select.ng process (S61 0) is cor W^™***^ from the regist er matrix 401 to the A 
multiplied result obtained during the pixei selecting ^^^SJ^^^to a ca.culation process- 
,atch 402. Next, in S614. objective data ('mage data l(x y) example) des.red l to be J ^ ^ . p B ^ 
ing is transferred from the register matnx 401 to the B latch ' ^ ^ e, f • ^ ^f"^ "Sective pixel position (x1, y1), the 
403 are multiplied in S616. As a result, at the Pressing element 400 of the p ^ ^ 

objective data is obtained as it is. On the other hand, at the "J"™^™^" 9 * S620, a desired processing 
is obtained. In S61 8, the multiplied result is stored ,n the , reg^ten ma nx 40T The reafte ^ ^ ^ 

at the desired address (x1 y1 ), to the control, 1 6 - ||ow the respec tive processing elements 
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pixels, the multiplied results obtained in S1 114 (Fin lOfCMh*™™ < - 

»n S61 6 (Fig. 1 3) as the muftiplied data. Accordingly in S ZZln ■ " ^ fhe ° bjeCtive is Stained 
processing onto the objective data at the Mm^!^^ SyStem 1 4 <™ Perform a desired 

101 52] As the calculation processinq performed in ^on wo 

ment operation to be described .ater, can beTet^ ^ Pr ° CeSSinS ° Perati ° nS such a * a " edge-enhance- 



(Example 3) 
[0153J 



E^ 0 ^ ,ca fc u.ated resu.t) from the register 

0154] in more concrete terms, a calculating-outputting SSJ„? predetermiaed calculations, 

•he image processing procedure S1 10 of Fig 8 (S650)can be performed as shown in Fig. 14 during 

[0155] It is noted that when the calculating-outputtinq process fSfinm ,c 

he mput image D has a.ready been transferred in S 04 IF* S ! tart6d ' ' ma9e da,a '<*■ » al ea <* pixel of 

401 of ,he — 9 ~ ^^TOJK^TOS 

enhancement process., can be employed. ca.cu.at.on, various processings, such as the edge- 

executed onto the objective pixel (xl yl) Asaresutt^m ! described already with reference to Fig. 10(C) is 

402. Next, the objective data (calculated result in S6 60) whfch s dl^Tf Z maWx 401 t0 the A ^ 

matnx 401 to the B latch 403 in S684. Then, the value * A te tc 40 and 5 t ™ n * TOd fr ° m ,he re 9 ist "" 

As a result, the calculated result is obtained as it is in the nZL , B ' atCh are ™"*«ed in S686 

(x1 , y1 ). Data of (0) . obtained at the other rema ming " "" position 

the register matrix 401 in S688. the multip.ied resuin 7£ZS t s^Tfr ^^ r6SUlt is stored in 

bus 155. The multiplied results of S686 are outoutted t«T ^ S K 69 ° from the out P"t address "28" to the output 
Accordingly, the calculated result at the ob ^ZuoOM IT, T 3 " ** Pr0CeSsin 9 elemen * 4^0. 

va.ue of the multiplied results obtained at ^iS^ rt J^ 0UtPU ^ t0 the C ° ntro ' circu * 15 « - 'ogica. OR 



(Example 4) 



data from the control circuit 1 5 to the processing efemenSo ! l TT'l * transfer/writin 9 Action to transfer 
The bi-directional data bus has both of: the ^SSS^SL ^ ?° '° the processi n9 elements 400. 
processing elements 400 and write data iJ^^SZ^S^l IT' **" fr0m the C ° ntro1 circu * 15 *> the 
outputting data from the processing elements 400 an^ransf- ™?h T ^"^'erring function for 

through a parade, processing procedure That h thl c Zl , -T lm " e (m1 X m2 pixe,s > under investigation 
position where a matching signal is obtainld ° ^ 1 5 °" COnfirm Within a sho * P^od of time the 

5J£! 2j TV ° f ima9e SSarChin9 ° peration wi " be d^cribed be.ow 

pattern P is .ocated within ^^L^^ZZTT^ ^ **' "™ " * dete ™ ed tha < 'h - r h 
have been reported as examp.es JXZS^^ 

Ployed. This algorithm determines that tw image aS idenL^ I ° f thS m ° St Simp,e a, 9 0ri,hms is 
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pixel (x.y) within the image D. 

ml n\2 



ERROH( P , q) - |j g feP + ± « + 3> ~ K (2> 

[0 1631 Fig 15 shows a flowchart of a searching process (S700) which is established based on this algorithm. The 
L of data processes wi.l be described below image proce ssing step S110 of Fig. 8. 

SSfTA2 m2) of the search image P is stored in the .e ^ ^ wil| be referred 

[0165] It is also noted that when the search.ng p ocess (S700 ,« started ^f™*^ J> ^ 104 R 8) from the 
!o as '".(p, q)» hereinafter) at each pixel in the ^eSSlSjSoM-lnfl element 400 (x,y) 

corresponding transfer shift register 410 to the I "some region in the register matrix 401 . 
(wh ich win be referred tc g"^^T»CSS?.n SZ3n» operation^ S70Z More spe- 

SS Er( P , q>. to 9 be stored in the register matrix ^ s n^at ding end position (1,1)) 

SSf dement 400 calculates an absolute^, of a d = o 

q) , at 'the subject pixe,, which is stored in some ^-J^^'SS Solute value to the value 
detection position of the search pattern. The P^JJ^^J^^ data l(p,q) and the image data P 

frauds 

m i"oT anTe emulated absolute va.ue -"f^ position (ml , m* in this 

[0169] in S705, the CPU 1 50 judges whether the detect.on pos.t.on ( . j) reaches the p ^ 
Le. the detection position (i, j) is still at the leading , end posfcon , U bu does ^1^^ ^ ^ 
S705). Accordingly, the program proceeds to S706. In S706. when . * nu y djrection . Thus, i is reset to 

When i = ml , it is determined that the detection pos,t,on reaches the trail ng end hn the x ( = ^ 

r-^a= 

bor^ 

[0170] When the transfer operation ,s completed thfl 'J^^ ™™^ ( 2 1)} within the search pattern P. The 
he memory 151 , image data P(i, j) at the present <^ p q) via the data bus 17, 18. In S704, 

CPU 1 50 transfers the retrieved image data to each process ng d £° ( e P ^ ™ he image data P(i, j), at the 
each processing element 400(p, q) calcu.ates the absolute ^^'^STT!) . at a pixel adjacent to the 

f 0 °m] n By heatedly performing the adding operation in S704 in this way, each processing element 400(p, q, can 

obtain a value ERROR(p.q). oerformed by successively transferring data to neigh- 

[0173] It is noted that the transfer operation in S707 may De penormeu y 

boring pixels in x and y directions. , ERROR(p.q) is obtained, 

[0174] P When the detection position V^^ST^ ^^^^ » < h ~ shold M ^ S7 ° 9 ' 
the program proceeds from S705 (yes in S705) to S708. In S708. me ^ 17 or 18. 

the CPU 150 transfers the set threshold va.ue E* to each P^^^^^ WTOR<P.««). that is < Er(P ' q) Wi,h 
[0175, in S710, each processing of ERROR(p.q) - threshold 

the threshold E th . More specifically, each processing eiemem ^uu^.m; 
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When Er(p.q) is greaterthan threshold value E th theprooram nr^JLl ^ ra ™ P r0Ceeds ,0 S711 ■ and outputs one (1). 
element 400 outputs the output data to both of he daS bu Tes iTZTrTroT ^ Zer ° (0) Each Droce ^ng 
*» the data buses 1 7 and 18 , ^ 

^ LgS ^tn^ bS iV^ -and on, a single data line 130y ou,u, 

^-SSSBKS! ,hat outputs one s ~ 

output the multiplied result to the output bus 155 In thfe L« 9 ! ' S710) by itS positional da 'a (P,q) and to 
output signal of one (1) as a judgingrasuTof f?o outZ^Z^ J"**'" 9 P ° S,t ™ ^ outputs an 
output values (0) that are the products of their ou tout XI Z ZZ * ^ A " ,h6 ° ther remainin 9 P«*s 
posmona. data (p,q)ofthe matching position will beTanste^ the posit.onal data ( P ,q). According, only the 
to this method, it » possible to transfer. tothec^ iss' According 

= ng calculated results, such as the absolute 

becomes one (1 ). search.ng range ,s successively decreased until the counted number 

d.g-tal converter 21 0 in correspondence ^JSJS^^^^ ' 3 ^ °™ 

array 1 1 In the parallel processing system 14, the processing «ln 9 * ^ r ° W ° f the P"otodetector 

w.th the photodetectors 120, and the shift registe^l^ 

photodetectors 120. The high-speed vision sensor *o f u^eMncTudef^t h °"? ,0 006 corres P° nd ence with the 
^Performing data transferring operation to the^ 

high-speed ,mage processings between the,r neighbor^ pixeTbC n^. I heprOCess,n 9 elem ^^ 400 can perform 

v "'»" s ™9a processings „„«, lne in(lul , » ag ^ °™^ e » (F, S S »-«).. processing can bs p.rtormaa 10 

ULs F :^ «- M ^ „ „ of ,„ e 

Ihe simptes, manners. a u.o.n.ightK.sng e0 ge entaSJL, k .17 ">«*<*» »>e edge-enhancement In one ol 

a^L,gnpc fl ^rL"::jrr- 

l'(x,y) = ll(x,y)-|(x-l,y)|. 

~Cof^ as a preprocess during the image 

S c z ssssrs °; 0 s ir e ~ ; ~ B executed as shown in Fi9 ' 1 6 before 

boring pixel. The'l is, the ^s^^Z^^Z^^ ' * *"* * « ™^ 

bonng prxels. The data l(x-1 , y) j £ st0 red in the reqistermaWx 401 ST ' ' V> ' tS mpUt tem ' mat for tne four ™»9h- 
lata l(x, y, and data .(x-1 , y) are transferred I from ^ 'raaS matriv ^TT' " 9 e ' ement 4 °° (X ' y >" Next ' in S1211, 
lively. In S121 2, the ALU 404 calculates ttTe X pL! h 1 . mt ° ,he A ,a,Ch 402 and 8 403. respec 
result of the ca,cu,a«on is 2 "T* ^ V) y) 31213 ^ 

of the calculation is read again into the A latch 40 in 1 2T4 In Z^l ,t I, n" Ca,CU ' ati ° n ' S C ° mpl6ted ' the result 
the difference. In S1216. the result of this c^^J^^^^^ - 
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[0186] The thus obtained two-neighboring edge-enhanced image ,'(x,y) is subjected to the image processing step 

5 image can be represented by the following equation: 

l'(x, y) = Kx, y-D + "(x, y+1 ) * Kx-1 , y) + Kx+1 , y) ■ 4i(x. y) . 

,0 [0188] When performing a four-neighboring smoothing, the image intensity value r<x. y) of a four-neighboring 
smoothed image is represented by the following equation: 

l- (x , y) = (4i(x, y) + I(x-1 , y) + I(x+1 , y) + l(x, y-1) + l(x, y+D) ' 8. 

subject algorithm is executed according to the present embodiment. 

Table 1 
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Name of Process 


Number of steps 


Time required (us) 


Two-neighboring edge detection 


1-bit input/ output 


5 


0.40 


Four-neighboring edge detection 


1 -bit input/ output 


11 


0.72 


Four-neighboring smoothing 


1 -bit input/ output 


14 


1.0 


Four-neighboring edge detection 


8-bit input/ output 


70 


5.6 


Four-neighboring edge detection 


8-bit input/ output 


96 


7.7 


Four-neighboring thinning 


1 -bit input/ output 


23 


r 1.9 


Eight-neighboring thinning 


1 -bit input/ output 


53 


4.2 


Convolution 


1-bit input 4-bit output 


40 


3.2 


Convolution 


4-bit input, 11 -bit output 


372 


30 


Poisson's equation 


1-bit input, 8-bit output 


63 


5.0 
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[0190] As can be apparent from Table 1 , according to the present ^SStESSS SS^S^ 
smoothing, thinning, convolution, correlation and ^ J^^H. »^ry «^-ion (FA) 

SET - noted that the .ength of each calculation tim -d ^%E£XZ 3£ 

shift registers 410 to transfer image data. It » also noted that the Version is 1 u s /bit, for 

of the ana.og-to-digi.al converter 21 0. If the .ength « ™J£££ paral.e. using the 1 28 

L sufficient* high resolution. It is noted that array 1 1 and 

128 photodetectors 120 or more are arranged. ^ f ^ 9 ^^" S !^ rdlngly eacn dev ice can be constructed with 
the parallel processing system ^^T^^^SS^L^. Additionally, the processing 

a high level of integration. It is therefore .potato to atta. ^. wjJhQ embodime nt, the processing 

^eS^^ 

^7^:^ - analog-to-digita, conversion rate per pixe. is 1 , s, 
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in one row into 6-bit 
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bit. It takes 0.768 ms (= 6 u s y 1 or\ - 

digita. signals (64 gradation^ £!££Tj^^ ft " n *" ' he 128 Pbotodetectors 120 
spondence with the Ph->S£^^ « P ™ idad * one^one^ 

calcu.ation processings can be completed withi o ^ it are operated in parallel. Accordingly, most 
calculating processes and the transfer pWCTiSS" f ' ° 1 ' Addi,ionall V. by performing the 
he respective processes and to shorten the overall processing t^e * ^ ° f dead time b «tween 

outputtmg a reset signal R after a desired number Xhave^lnZ^ ° f ana, 09-t°-d<gital conversion, by 
an analog-to-digital conversion of the next optica [2*2 ,T, conv erted so that the process will shift to perform 
a higher rate of speed. For example, in Z!!S!^'^SSZ " * * """P""** processes" 

to process images in two levels, that is. one btt ^i.^^^irT ,ng " 3 *** ^ °' Spee * 1 is P referable 
about 1/6 the time required for transferring six bits ^^£?S? T T * 8h0rtenad * 0128 ms - whic " « 
feedback operation. Converse.y, when the 'object is ir^SS^S^S T t0 * a °° n ™ 0f 
object with a higher precision 9 Wl * the Sradat.on can be raised to be able to track the 

* r 3 — « * - necessary 

that a shift register line, for transferring data « a nTS'.ength of 8 b IJTi the . 10 3 ^ * iS " 0ted 
sh,ft reg.s.ers that has the total fixed .ength equal to Z p roSof eioh h" ' " COnStructed fro ^ one set of 

row. In the one set of shift registers, a plurality^ e bf-shW Tea sters which * l ° ta ' (N1) ° f P * elS in ona 

of which has eight bits, function as individual shift Listen , £ J? T SPaCed fr ° m ° ne anotner and ea <* 

■f the brt .ength of the image data for each pixel is netted to etaht Zt h COrres P° ndi "9 pixels. Therefore, 

worth of image data correctly to the sh,ft register at the ^^SS^T c ' mP ° SSib,e * e3Ch P ixel ' s 

image data is added with a dummy signal to have eight ^^7^^ °" ° rthiS reaS ° n ' eacn P ixel ' s w °rth of 
registers. It can be ensured that imag^data wi.fbe X^^ST "~ *"* iS *— * *» 

-veL^™^ 

digital converter 210 in correspondent ^^^^JT 0 ^^ ana " 13 has one analog to-' 

array 1 , ,„ the paralle( prQC J » tlpZ^^J!^ ^ t0 each ™ °< the photodetector 

wrth the photodetectors 120, and the shift registers 4 He ^ 

Photodetectors 120. By providing a processing elemem 400 ZZrr^n T ^ * ° m corres P°" d once with the 
poss.bleto perform, at a high speed image oroces JnTrltft correspondence with each photodetector 120. it is 
[0197] Additionally^ 

of transfer lines, provided between the photodetecfo^ 1 20 and the o^«^ T ' S P ° SSiD ' e t0 r6dUCe the number 
f.gurat,on where an ana.og-to-digital converter t ™ ? em6ntS 4 °°' in ^P™ to a con- 

photodetectors 120 and the processing elements 400 can be nr.H f P ° ndenCe Wit " 9ach P h °'ocetector 120. The 
Accordingly, both components can be n^SZ^^S^^ r^o -* * ted 
v.s.on sensor 10 having a large number of pixels cZ bTsZ^ul* T ° f ,ntegrat,on - Production of the high-speed 
converter210isprovidedfor each row the overalt^oc efi !innT P 1 S n0,ed that because ° ne analog-to-dLal 
most processes can be completed wiJn« 

FA robot control, are processed at a 64 gradation Such a Stt n P * e ' ' mageS ' Wnicn are sufficie "t for 

h.gh-speed vision sensor 10 of the present embodLpn 0 ^ n ' 9 , h - s P eed Processing can be attained. Accordingly the 
perform basic ,mage captions at aKgi Ipe^ ^ * C0 ^ uc ^ ^ a simple circuit, but afso'can 

Prm/a^ 

mi oo, US iS h theref ° re posstole 10 perf0 ™ ^ -SS S32, " 9 6 X d ' reCti0n data bus 30(1 the v directi °n 

Ucess ~ 

transfer process efficiently. Because the ca.cu at^n prcless an^t" 9 ' & 9ChieVin9 Ca ' CU,ati ° n Pf ° CeSS a " d 
possible to reduce wait time between the respeS Z processed and to ^ Pr ° CeSS ^ h parallel ' " is 

speed. That is, i, is possib.e to attain a function to perfom Satton D » P 'T 96 pr0CeSsin 9 at a ^ster overa.l 

from each other by using the transfer shift register^ t /t^sfe datlTromTh a " d ,ranSfer Pr ° CeSSeS inde Pendent.y 
ing elements. A real-time operation can be attuned analog-to-digital converters to the process 

foTabo^^^ 

Snt^ ig ^^^ Offers 1 9 and 20 are provided. However, ■ a 

unnecessary to provide the data buffers. 15 and tne da,a buses 1 7 and 18. it is 
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in a passing alaman, 400(1 . v) .hat ,s local a a la djng adga o! a p , ow , s , (ansmitIea 

Losing al.man.a 400 in additon ,o tha o.ldala.ing t.m. ,n TaWa « » naiad Iter aa daacnWC I aw V 9 

U»al o'Larpidn » pa, J- » ]M£ £ ' n p SS d™*, a, »»,a £ grada.ian). On ,na 
input signals, outputted from all the 1 28 photodetecto s t ^ in < ane . * Accordingly, even 

r rsssrsss sssr is ssss* - — — , a* 

ever, the analog-to-digital converter 210 and * e ^^"^^JP „ Th e amp array 12 has N2 charge 
shown in this figure, an amp array 12 ,s connected to the P^S^mT^ anj 12 and the parallel processing 
amps 221 . The ana.og-to-digita. converter array 1 a 21 0 . With this construction, 

system 14. The analog-to-digital converter array 13 '^^^M Irom^ie N1 photodetectors 120 on the 

parallel processing system 14. calculate- the total sum of image intensities, tfce 

[0205] Additions, in the ^^^^^^^T^ ^ ca(cu|at|0n process of Fig . 
^Z^TZ^oZ 1 si cT^e added with a circuit for ca,cula ti ng a total sum of outputs from 
£S£SS£n«* 400. This circ.it may be used to calcu.ate the total sums. 

30 INDUSTRIAL APPLICABILITY 

[0206, The high-speed vision sensor of the present invention can be emp.oyed in a wide variety of vision recognition 
processes, including FA robot control. 
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Claims 

1 . A high-speed vision sensor, comprising: 

, pnotodatao.o. a™, na»ing a plaral»» 0. pno,oda,ac,Oda. «** and arangad mo-ddnensionan, in a P'oraldy 

ted from the photodetectors in the corresponding row, „ r „ v th arallel processing element 

a parallel processing system ^^^^^J^^^S^^ a plurality of rows 
array having a plurality of processing elements of pnotodete ctors in the 

and in a plurality of columns and in one-to-one J^^f^^^^^JSdlgW.lgnal. which 
photodetector array, each processing element performing a predetermined calculation g 
are transferred from the analog-to-digital converter array; djrection data -transfer data lines which 
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